Generalized Kuhn- Tucker Conditions for 

N-Firm Stochastic Irreversible Investment 

under Limited Resources* 

. . . Maria B. Chiarolla t Giorgio Ferrari* Frank Riedel§ 

g ; February 27, 2013 

Abstract. In this paper we study a continuous time, optimal stochastic investment problem 
under limited resources in a market with N firms. The investment processes are subject to a 
time-dependent stochastic constraint. Rather than using a dynamic programming approach, we 
exploit the concavity of the profit functional to derive some necessary and sufficient first order 
conditions for the corresponding Social Planner optimal policy. Our conditions are a stochastic 
\*J . infinite-dimensional generalization of the Kuhn- Tucker Theorem. As a subproduct we obtain an 

enlightening interpretation of the first order conditions for a single firm in Bank [5]. 
r*] ' In the infinite-horizon case, with operating profit functions of Cobb-Douglas type, our 

method allows the explicit calculation of the optimal policy in terms of the 'base capacity' 
process, i.e. the unique solution of the Bank and El Karoui representation problem [JJ. 

Keywords: stochastic irreversible investment, optimal stopping, the Bank and El Karoui 
Representation Theorem, base capacity, Lagrange multiplier optional measure. 

MSC2010 subsject classification: 91B70, 93E20, 60G40, 60H25. 

IT) ■ JEL classification: C02, E22, D92, G31. 

en 

m ! 1 Introduction 

O 

In the last years the theory of irreversible investment under uncertainty has received much 
attention in Economics as well as in Mathematics (see, for example, the extensive review in 
Dixit and Pindyck [15j). From the mathematical point of view, optimal irreversible investment 
rS ■ problems under uncertainty are singular stochastic control problems. In fact, the economic 

C3 constraint that does not allow disinvestment may be modeled as a 'monotone follower' problem; 
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that is, a problem in which investment strategies are given by nondecreasing stochastic processes, 
not necessarily absolutely continuous with respect to the Lebesgue measure as functions of time. 
The pioneering papers by Karatzas [20], Karatzas and Shreve [2TJ, El Karoui and Karatzas [16] 
(among others) started the application of 'monotone follower' problems to Economics. These 
Authors studied the problem of optimally minimizing a convex cost (or optimally maximizing a 
concave profit) functional when the capacity is a Brownian motion tracked by a nondecreasing 
process, i.e. the monotone follower. They showed that any such control problem is connected to 
a suitable optimal stopping problem whose value function v is the derivative of the value function 

V of the control problem. Moreover, the optimal control v* defines an optimal stopping time 
t* by the simple formula r* := inf{£ £ [0, T) : v*(t) > 0} A T. Later on, this kind of link 
has been established also for more complicated dynamics of the controlled diffusion; that is the 
case, for example, of a Geometric Brownian motion (Baldursson and Karatzas [TJ), or of a quite 
general controlled Ito diffusion (see Boetius and Kohlmann [8], Chiarolla and Haussmann |12j . 
Chiarolla and Ferrari |13j , among others) . More recently, Boetius [9] , Chiarolla and Haussmann 
[10] and |11| . and Karatzas and Wang [23] showed that such connection holds in the case of 
bounded variation stochastic control problems as well; the value function of the control problem 

V satisfies Jj V = v, where v is the saddle point of a suitable Dynkin game, that is a zero-sum 
optimal stopping game. 

In the last decade several papers handled singular stochastic control problems of the mono- 
tone follower type by deriving first order conditions for optimality. That is the case, for instance, 
of Bank and Riedel [2] in which the Authors studied an intertemporal utility maximization 
problem with Hindy, Huang and Kreps preferences, of Bank and Riedel [3] in which the optimal 
dynamic choice of durable and perishable goods is analyzed, or of Riedel and Su [27] in which 
a very general irreversible investment problem with unlimited resources is treated. In these 
papers the optimal consumption, or investment policy, is constructed as the running supremum 
of a desirable value. Such level of satisfaction is the unique optional solution of a stochastic 
backward equation in the spirit of Bank-El Karoui (cf. [3], Theorem 3) and may be represented 
in terms of the value functions of a family of standard optimal stopping problems. 

The link between irreversible investment problems and optimal stopping is also relevant in 
Economics. In fact a firm operating in a market with uncertainty not only has to decide how 
to invest but also when to invest. The optimal timing problem is then related to option theory, 
since it may be viewed as a 'real option', an option whose strike price is the cost of investment. 
It follows that exercising a real option means to invest properly at an optimal time. 

The investment problem becomes even harder if one takes into account the fact that the 
available resources may be limited. The problem turns into a 'finite fuel' singular stochastic 
control problem since the total amount of effort (fuel) available to the controller (for example, 
the firm's manager) is limited. The mathematical literature on this field started in 1967 with 
Bather and Chernoff [U] in the context of controlling the motion of a spaceship. Finite fuel 
monotone follower problems were then studied by Benes, Shepp and Witsenhausen in 1980 [7J. 
In 1985 Chow, Menaldi and Robin [T5] and Karatzas [22] used a PDE approach and purely 
probabilistic arguments, respectively, to show that the optimal policy of a 'monotone follower' 
problem with constant finite fuel is 'follow the unconstrained optimal policy until there is some 
fuel to spend'. Much more difficult is the case of finite fuel given by a time-dependent process, 
either deterministic or stochastic. 

In 2005 Bank [5], without relying on any Markovian assumption, generalized the optimal 
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policy proposed by Karatzas [22] to the case of a stochastic, increasing, adapted finite fuel 
process 9(t). The Author characterized the optimal policy of a cost minimization problem as 
the unique process satisfying some first order conditions for optimality (cf. [5], Theorem 1), 
'the optimal control should be exercised only when its impact on future costs is maximal; on 
the other hand, when the cost functional's subgradient tends to decrease, then all the available 
fuel must be used\ More in detail, if §(V) is the Snell envelope of the total cost functional's 
subgradient V ' u C(v) (i.e., S(u)(t) := essinf t < T < T E{V„C(i/)(r)|J r i }), and M{v) + A(v) is its 
Doob-Meyer decomposition, then Bank [5] proved that v* is optimal if and only if 

(i) v* is flat off {V u C(v*) = S(^*)}, 

(ii) A(y*) is flat off {z/* = 9}. ^ ' ' 

Moreover, the Author constructed the optimal control v* in terms of the 'base capacity' process, 
i.e. a desirable value of capacity. Mathematically such process is the unique optional solution 
of the Bank-El Karoui representation problem 0] . 

In this paper we generalize Bank's single firm problem to the case of a Social Planner in a 
market with N firms in which the total investment is bounded by a stochastic, time-dependent, 
increasing, adapted finite fuel 9(t); that is, the case X^i=i v (*) — ^(*) P-a.s. for all t G [0, T]. 
The Social Planner's objective is to pursue a vector v^ € R^ of efficient irreversible investment 
processes that maximize the aggregate expected profit, net of investment cost, i.e. 



sup 



Vfif / e~ 5{t) R {i) {X(t),v^{t))dt- I e- s ®dv®(t)\. (1.2) 






Here the operating profit function BS l > of firm i, % = 1, 2, ..., N, depends directly on the cumula- 
tive control exercised since we do not allow for dynamics of the productive capacity. As in Kobila 
[25] , and Riedel and Su [27] , the uncertain status of the economy is modeled by an exogeneous 
economic shock {X(t),t & [0, T]}. Although our finite fuel 9 is increasing as in Bank [5], his 
results cannot be directly applied to each firm since for each i the investment bound 9— X^j v 
is not an increasing process. To overcome this difficulty we develop a new approach based on a 
stochastic generalization of the classical Kuhn- Tucker method. That is accomplished as follows. 
By applying a version of Komlos' theorem for optional random measures (cf. Kabanov [19j . 
Lemma 3.5) we prove existence and uniqueness of optimal irreversible investment policies. Then 
we use the concavity of the profit functional to characterize the optimal Social Planner policy as 
the unique solution of some stochastic Kuhn- Tucker conditions. The Lagrange multiplier takes 
the form of a nonnegative optional random measure on [0, T] whose support is the set of times 
for which the constraint is binding, i.e. when all the fuel is spent. Hence, as a subproduct we 
obtain an enlightening interpretation of the first order conditions that Bank [5] proved for a 
single firm optimal investment problem. In fact, we show that process A{y*) in (jl.ip is equal to 
the Lagrange multiplier of our control problem. As expected in optimization under inequality 
constraints, our Lagrange multiplier A grows only when the resource constraint is binding (see 
our equation (|3.9|) below). This condition corresponds exactly to Bank's ([l.ip -(ii). 

When the N firms have operating profit functions of Cobb-Douglas type, with a different 
parameter for each of them, our generalized stochastic Kuhn- Tucker approach allows for the 
explicit calculation of the Social Planner optimal investment strategy. Such optimal policy is 
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given in terms of the 'base capacity' process, i.e. the unique solution of the Bank-El Karoui 
Representation Problem [3]. Finally, when the finite fuel is constant, we recover two classical 
monotone follower problems for which we are able to identify the compensator part in the 
Doob-Meyer decomposition of the profit (cost) functional's supergradient (subgradient) as the 
Lagrange multiplier of the optimal investment problem. 

The paper is organized as follows. In Section [2] we set the model. In Section [3] we introduce 
the generalized stochastic Kuhn- Tucker conditions for the Social Planner problem. Finally, in 
Section [3] we solve an iV-firm Social Planner optimization problem and we test our approach on 
some 'finite-fuel' problems from the literature (cf. [5] and [22] . among others). 

2 The Model 

We consider a market with TV firms on a time horizon T < +oo. Let (Q, J-, {J~t}te\oT\ >^) 
be a complete filtered probability space with the filtration {J"t,t G [0, T]} satisfying the usual 
conditions. The cumulative irreversible investment of firm i, i = 1,2, ...,N, denoted by v^ l '(t), 
is an adapted process, nondecreasing, left-continuous, finite a.s. s.t. z/*)(0) = yW > 0. 

The firms are financed entirely by equities but we focus primarily on the irreversibility of 
investments and do not model precisely the rest of the economy. It is reasonable to assume that 
the firms cannot invest in natural resources as much as they like. In fact, we assume that the 
total amount of natural resources available at time t is a finite quantity 9(t); that is, 

N 



$^" (i) (t)<0(*), P-a.s., for t€[0,T\. (2.1) 



The stochastic time-dependent constraint {9(t), t G [0, T]} is the cumulative amount of resources 
extracted up to time t. It is a nonnegative and increasing adapted process with left-continuous 
paths, which starts at time zero from 8(0) = O > 0. We assume 

E{d(T)} < +oo. (2.2) 

We denote by Sg the nonempty set of admissible investment plans, i.e. 

<Sd '■= {k '■ x [0,T] — > M + , nondecreasing, left-continuous, adapted process s.t. 

v 
z,W(o) =y ( i ) ) P-a.s., i = l,2,...,N, and ^ v {€) {t) < 6(t), P - a.s. Vt G [0, T]}. 

i=i 

Let {X(t),t G [0, T]} be some exogenous real-valued state variable progressively measurable 
with respect to J-j. It may be regarded as an economic shock, reflecting the changes in tech- 
nological ouput, demand and macroeconomic conditions which have direct or indirect effect on 
the firm's profit. At the moment we do not make any Markovian assumption. 

We take the capital good as numeraire, hence we express profits, costs etc. in real terms, 
not nominal ones. Hence the price of a unitary investment is equal to one. We take the point 
of view of a fictitious Social Planner aiming to maximize the aggregate expected profit, net 
of investment costs, Jsp{e) ( see equation (|2.5p below), by allocating efficiently the available 
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resources. We denote by S(t) the Social Planner discount factor. S(t) is a nonnegative, optional 
process, bounded uniformly in (co,t) G fl x [0, T]. Assumption (|2.2p ensures 

e( [ e~ m dv^it)] <+oo, i = l,2,...,iV, (2.3) 

I J[0,T) J 

i.e. the investment plan's expected net present value of firm i is finite. 

The operating profit function of firm i is By' : M x M + — > K + , i = 1, 2, ..., N. At time £, when 
the investment of firm i is v^'{t), Ry> (X (t) , v^ 1 ' (t)") represents the amount of goods produced 
by firm i under the shock process X(t). The Social Planner problem is 



where 



V SP := sup Jsp(e), (2.4) 

veSg 



N 
Jsp(k):=Y.^ (1) ) ( 2 - 5 ) 



and, for i = 1,2, ...,iV, 



J^«)=e{ / e" 5 W i?W(x(t),i/W(t))dt- / e-'(*W*)(t) }. (2.6) 

I JO J[0,T) J 

Notice that Ji{v^ l >) is the expected total profit, net of investment costs, of firm i when the Social 
Planner picks u&Sg. 

The operating profit functions satisfy the following concavity and regularity assumptions. 

Assumption 2.1. 

1. For every x € M and i = 1,2,...,N, the mapping y — > R^ l '(x,y) is increasing, strictly 
concave, with continuous decreasing partial derivative Ry (x,y) satisfying the Inada con- 
ditions 

lim flW (x, y) = oo, lim i# ( x > 2/) = °- 

2. R® {X(uj,t),v®(uj,t)) is dF (g> dt-integrable, for i = 1,2,..., N. 

3. The process (u},t) — > sup R^ l \x(oo,t),u ( - l \uj,t)) is dF®dt-integrable, for i = 1,2, ...,N . 

i/M(o;,t):i/<ESg 

Under (|2.2p and Assumption 12. II the net profit Ji{v^ % ') is well defined and finite for all admissible 
plans. 

In the next Section we show how to handle constraint (|2,ip in order to find the solution to 
Social Planner problem (|2.4p . 
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3 A Stochastic Kuhn- Tucker Approach 

In this Section we aim to find an optimal investment plan by means of a gradient approach. As 
in [27], proof of Theorem 2.6, by applying a suitable version of Komlos' Theorem for optional 
random measures (cf. [19], Lemma 3.5) we obtain existence and uniqueness of a solution to 
problem (|2.4|) . In fact, Komlos' Theorem states that if a sequence of random variables (Z n ) ng ^ 
is bounded from above in expectation, then there exists a subsequence (Z nk )k^n which converges 
in the Cesaro sense to some random variable Z. In our case the limit provided by Komlos' 
Theorem turns out to be the optimal investment strategy. 

Theorem 3.1. Under \2.2\i and Assumption \2. 11 there exists a unique optimal vector of irre- 
versible investment plans v^ G Sg for problem \2.J$ . 

Proof. Let v_ G Sg and denote by H the space of optional measures on [0, T]. Then, the invest- 
ment strategies i/W may be regarded as elements of H, hence Sg C H n . 

Let (K n )n£N be a maximizing sequence of investment plans in Sg, i.e. a sequence such 
that lim Jsp{E_ n ) = Vsp- By (|2.2|) we have that the sequence (E{f„ (T)}) ne N is bounded 

n— >oo 

for i = 1,2,..., N; in fact, E{u^(T)} < E{6(T)} < oo. By a version of Komlos' Theorem for 
optional measures (cf. [TO], Lemma 3.5), there exists a subsequence (£ n )neN that converges 
weakly a.s. in the Cesaro sense to some random vector v^ G H . That is, for i = 1, 2, ..., N, we 
have, almost surely, 

1 n 
I®(t):=-Y,i>j\t)^i'*(t), as ™^oc, (3.1) 

71 j=0 

(i) 

for every point of continuity of ui , i = 1,2, ..., N. Notice that y_ n G Sg for all n implies that 
also the Cesaro sequence I_ n belongs to Sg due to the convexity of Sg, hence J2i=i K (t) < #(£)> 
for n G N. It follows that, almost surely, 

v 
J>J° (<)<*(*), (3-2) 

i=\ 

which means v_^ G Sg. 

Since (y n l )neN ls a maximizing sequence so is (In )neN by concavity of the profit functional. 
Then Jensen inequality and the dominated convergence theorem yield 



1 
Jspfa) > lim - Y^Jsp(kn) = Vsp- (3.3) 

j=0 

Finally, uniqueness follows from the strict concavity of the Social Planner profit functional. □ 

We now aim to characterize the Social Planner optimal policy as the unique solution of a 
set of first order generalized stochastic Kuhn- Tucker conditions. Notice that the strict concave 
functionals Ji, % = 1, 2, ..., N, admit the supergradient 



v 
fort G [0,T]. 



V v Ji{u®)(t):=E\ I e- s ^R^(X(s),u^(s))ds T t \ - e~^\ {t<n (3.4) 
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Remark 3.2. The quantity V ' yJi(v( l >)(t) , i = 1,2, ...,iV, may be interpreted as the marginal 
expected profit resulting from an additional infinitesimal investment at time t when the invest- 
ment plan is u^' . Mathematically, V y Ji{v^ % ') is the Riesz representation of the profit gradient at 
v^ 1 '. More precisely, define V yJi{v^ % ') as the optional projection of the progressively measurable 
process 

$i(",t) ■= J e- 5 ^Rf{X{u,s),^{u,s))ds - e~ s ^H {t<T} , (3.5) 

for lo £ Q,, and t £ [0, T]. Hence V y Ji{v^ 1 ') is uniquely determined up to P ' -indistinguishability 
and it holds 

A [ VyJi(v®)(t)dv®{t)\ =^\ [ $i(t)d»®(t) 

I J[0,T) J I J[0,T) 

for all admissible u^'(t) (cf. Theorem 1.33 in li£$). 

3.1 Generalized Stochastic Kuhn- Tucker Conditions 

Let £>[0, T] denote the Borel <7-algebra on [0, T]. Recall that if /?(£) is a right-continuous, adapted 
and nondecreasing process, then the bracket operator 



(at,P) =^< / a(t)dP(t) } (3.6) 

I J[0,T) J 

is well defined (possibly infinite) for all processes a(t) which are nonnegative and Tt £3 £>[0, Im- 
measurable. Notice that the bracket is preserved when we pass from a to its optional projection 
«(°) (cf. 0S|, Theorem 1.33); that is 

(a,P) = {a(°\p). (3.7) 

Since the constraint is 9{t) - '£*L 1 ^{t) > 0, P-a.s. for all t G [0,T] (cf. flZU)), we define 
the Lagrangian functional of problem (|2.4[) as 

v 
C 6 {v,X) = Jsp(K) + (0-J2" {i) ^) 

i=l 
N ( rT 



= Vl{/ e~ s ® R®(X(t),u®(t))dt- [ e~ m du^(t)\ (3.8) 

~t Uo J[o,T) J 

+ e{ / [*(*)- X>«(t)]dA(t)), 

I J[0,T) ~y J 

where d\(uj, t) is a nonnegative optional measure, which may be interpreted as the Lagrange 
multiplier of Social Planner problem (|2.4p . By using Fubini's Theorem we write the bracket 
(9 — X^j=i v ) X) in a more convenient form, that is 

N f V . 

<0-J>«A)=E / [0(t)-J>«(t)]dA(t)[ 
i=i l J l°> T ) t=i J 

/ \f (d0(a)-Vdi/W(a))ldA(t)l+ Ke{ I d\{t)\ 

J\0,T) l J\0,t) "H J J I J\0,T) J 
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E 



[0,T) 



N 



[t,T) 



d\(s) (d6{t)-J2 d,yii) (t)) \ + KE 



i=l 



[0,T) 



d\(t)\, 



where K := O - Y^Li V W = 9 (°) ~ E£=i ^ W (0). Hence 

v 

N r r T 



Ve( / e _<5(t) Rfi\x(t),v^(t))dt- I e- m du^(t)\ 

~y I JO J[0,T) J 



+ E 



[0,T) 



A' 



dA(s) (d0(i) - Vdi/W(t)) \ + KE{ / dA(t) }. 

[t,T) J ~ J I 7[0,T) 



We now obtain stochastic Kuhn- Tucker conditions for optimality with a stochastic Lagrange 
multiplier process that takes care of our dynamic resource constraint. A similar approach may 
be found in [2] for an intertemporal utility maximization problem under a constant badget 
constraint, with Hindy, Huang and Kreps preferences. 

Theorem 3.3. Under Ii2.2\) and Assumption \2.1l an admissible investment vector v_^ is the 
unique solution of the Social Planner problem \2.J$ if there exists a nonnegative Lagrange mul- 
tiplier measure dX(u,t) such that E{ Jr 0T \ dX(t)} < oo, and the following generalized stochastic 
Kuhn-Tucker conditions hold true for i = 1,2, ...,N 



V y Ji(u^)(t) < Ei I dX(s) 

l[t,T) 



Tt 



[0,T) 



,«> 



[t,T) 



Tt 



a.s., Vie [0,T), 



dui i] (t) = 0, 



a.s., 



(3.9) 



e( / [0(t)-Yu®{f)]d\{t)\ =0. 



Proof Let z^ satisfy the first order Kuhn-Tucker conditions (|3.9p and let z/ be an arbitrary 
admissible plan. By concavity of RS l >(x, •), i = 1, 2, ..., TV, and Fubini's Theorem we have 



A ; 



^sp(^) - Jsp(^) 



E E 

8=1 



V 

> £ E 
j=i 



<*)[&) (X(t),v?\t)) - R®{X(t),v®(t))]dt 

- / e-«Wd(^(t)-i/«(t)) 
iro.D 



- St R®(X(t),4\t)) (4\t) - v^(t))dt 
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J[0,T) 



N r r r T 

J^EJ / / e- s ®R®(X(t),u?\t))dtd(v?\8)-v®(8)) 



i=X 



N 



(3.10) 



«=i 



-*Wd(i/^ (a)-i/« (a)) 

[0,T) 



Ve<| / v^H(*)^(*)-^ w (*)) 

[0,T) 



Now ()3.9p implies 



Jsp(}>*) ~ Jsp(v) > J2 M \ I V v J > 



,(*h 



,(') 



V 

> j> 



(^)(t)^r(t)-^W) 



.w, 



E<; / dA(s) ^ } d(v?>(t) - i/«(i)) } (3.11) 
"—[ I i[o,r) I J[t,T) 

N 

d\(s)]d(u?\t)-vW(t)) 

[o,r) L i[t,T) 



i=l 



and the nonnegativity of d\(t), the admissibility of u, and another application of Fubini's The- 
orem give 



N 



Jsp(k*) - Jsp(k) > X> 



i=l 



[0,T) 



[*,r) 



,(') 



dA(s) d(i/i ,J (t)-i> w (t)) 



D 



= E / J^[^(t)-i/W(t)]dA(t) 

= e{ / [*(*)- f>«(t)]dA(t))>0, 

where the last line follows from (|3.9|) . third condition. 

Conditions (|3.9[> are also necessary for optimality under the assumption that 

rT 
uj^6{uj,T) B?>(X(u,t),e(u,T))dt is dP - integrable, i = 1,2,..., JV. (3.12) 

Jo 

The proof is based on arguments similar to those used in the finite-dimensional Kuhn- Tucker 
Theorem. Denote by T the set of all stopping times in [0,T], P-a.s., and notice that 



>[r,T) 



F T 
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for every i = 1,2, ...,N and for all r € 7" '. In fact, if not, then there would exist some r G T 
such that V yJiiyi )(r) > E{Jj_ T .. c?A(s)| JV} which, togheter with the continuity of R y and 

the linearity of investment costs, would imply that a sufficiently small extra investment at r is 

(i) 
profitable and hence contradict the optimality of z/* , i = 1, 2, ..., iV. 

In the next Lemma we show that under ()3.12j) the optimal policy v^ solves the linearized 

problem 

sup fy\ f $*(s)dvV(s)\ (3.13) 

E^S e ~{ I J[0,T) J 

where $* is the progressively measurable process associated to VyJity* ), i = 1,2, ...,7V, and 
defined in (|3.5p . Solutions of the linear problem will then be characterized by some 'flat-off 
conditions' in the second Lemma. 

Lemma 3.4. Let v^ he optimal for problem \2.1$ and assume 113. 12\) . Then it solves i3.13\) . 

Proof. Let v_ be an admissible plan. For i = 1, 2, ..., N and e e (0, 1), set i4 = ev^' + (1 — e)v* 

and let <3?^ be the progressively measurable process defined in (|3.5p associated to V^tTi^e ). 

Then lim e ^ ^(i) = »^(t), p -a-s., as well as lim e ^ *|(t) = $*(*), P-a.8., by continuity of flj?. 
Optimality of Z/*, concavity of y — >• R^'(X(t),y) and Fubini's Theorem, imply 

> - [JspCke) - JspiiU)] 

6 i=i L Jo 

- s ^d(u^(t)-uP(t))\ (3.14) 

> f>{ I m)d(^(t)-4\t))\, 

~1 l J\0,T) J 



'[0,T) 

V 



since e(i^ ; — i/* J = f e — i>* . 
In order to prove that 



f>{ / *?(t)d(i/«(t) - «/«(*))) < o 

~f I 7fO,T) J 



i=l ^ i[0,T) 

we need to apply Fatou's Lemma to conclude (by (|3.14|) ) 

f>{ / *J(t)d(i/W(*)-i/W(t))j<liminfX) E { / *K*)^ (0 (*)-^°(*)))<0. 

To check the hypothesis of Fatou's Lemma, we must find tflP-integrable random variables, Gi(u), 
i = 1, 2, ..., N, such that 

If(u) := f ®l(u,t)d(v {i) (u),t)-v!?\u,t))>G i (u), wen, ee (0,1). 

i[0,T) 
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We write I\ as 



If 



e- s ®R®(X(t),u®(t))(u®(t) - vX\t))dt - / e- s ®d(u®(t) - u?\t)) (3.15) 

J[0,T) 



.W, 



by Pubini's Theorem. Then, from concavity of R> l >(x, ■) and 

' < i/W(i), on {t : i/W(t) - i/^(t) > 0}, 



"i°(*)< 



(3.16) 



[ >i/W(t), on{t:i/W(t)-i/i' ; (<) <0}. 



(0/ 



.«/ 



we obtain 



II > / 

'o 

./o 



s ^R^\X(t),^\t))(^Ht)-ui l \t))t {i/{l) ^ )} (t)dt 

«WflW( X (t) fV W(t)) ( V »( t ) -»/«(*)) l {j/(0<v w } (t)(ft 
e-^dCi/W^)-!/®^)) 

WflWpTCtJ.i/WCt)) (^ (i) (i) - i4°(t))dt 
e-'Wd(i/«(t)-i/i°(t)) 



[0,T) 



[0,T) 



r [0,T) 



Hence we define 



Gi(u):= I VyJi(v®)(u,t)d(v®(u,t)-ujP(u,t)), u € «, » = 1,2,..., TV. 

'[0,T) 



(3.17) 



Now (|2.2p . Assumption 12. II and condition (|3.12|) . imply the integrability of Gi{uj) since |Gj(cj)| < 
C[0(w, T) + (1 + 0(w, T)) J T i? (i) (X(w, t), 0(w, T)) dt], lo G fi, with C a constant. D 

Lemma 3.5. Lei fi, i = 1,2, ...,N be optional processes and define 

H(s) := max {/+(«), /+(*), ..., /+(*)} . (3.18) 

Then every solution z> to the linear optimization problem 



sup Ve( / fi(s)dv^{s) 
K&Se i=1 I J[Q,T) 



satisfies the 'flat- off conditions' 



I (f i (s)-n(s))di> (i Hs)\=0, i = l,2,...,N. 

r [0,T) 



(3.19) 



(3.20) 
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Proof. Obviously 

f>{ / /<(*)&/«(*)) <f>( / M(s)^«( S )l. (3.21) 

The equality holds if and only if y_ satisfies (|3.2U|) . In fact (|3.2U|) implies the equality. Conversely, 
if equality holds in (pOT]) . then Ya=\ e { J[ ,t)(/*( s ) ~ /"( s )) dv®(s)} = 0. Hence ([3T20]l follows 
from the fact that the integrands are nonpositive. D 

Remark 3.6. We point out that our stochastic Kuhn-Tucker approach may be generalized to the 
case of investment processes also bounded from below by a stochastic process. In that case the 
Lagrangian functional is defined in terms of two Lagrange multipliers, d\\(uj,t) and d\2(w,t). 

4 Applications of the Kuhn-Tucker Conditions 

In this Section we test our approach on some 'finite-fuel' problems from the literature (cf. [5] 
and [22], among others) and we solve a iV-firms Social Planner optimization problem. In the 
following examples we assume 5(t) = St, with 5 > 0, and T = +oo. 

4.1 The Finite Fuel Monotone Follower of Bank [5] 

In the setting of Section [21 under (|2.2p and Assumption 12.11 we take N = 1 and T = +oo. We 
set v := z/ 1 ), y := y^ , R := R^ and J := J\. Notice that with 

c(uj,t,u(oj,t)) := -e~ 5t R{X(uj,t),v(uj,t)) : 

and instantaneous cost of investment 

k(u,t) :=-e- St , 

we recover Bank's model [5]. Recall that Bank's optimal investment (cf. [5], Theorem 2) was 
given by 

u m (t):= sup (l(s) A 6(s)) V y (4.1) 

0<s<4 

in terms of the 'base capacity' process l(t) (cf. [27] for this definition) which solves uniquely the 
stochastic backward equation (cf. [1], Theorem 3) 

e{ I e~ 5s R y (X(s), sup l(u))ds TV 1 = e" 5r , WeT. (4.2) 

I Jt t<u<s ) 

When l(t) is a continuous process we show the optimality of v*(t) by means of our Generalized 
Kuhn-Tucker conditions; as a subproduct we obtain an enlightening interpretation of the first 
order conditions stated in [5], Theorem 1, for a single firm optimal investment problem. Notice 
that continuity of l(t) is guaranteed when the shock process X{t) is continuous as well, as in the 
case of a diffusion (cf. [27], Theorem 6.5). 
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Recall that the supergradient of the net profit functional is the unique optional process given 



by 



V y J(u)(t):=El ! e- Ss Ry{X(s),u(s))ds 
By Theorem 13.31 an investment plan v* (t) is optimal if 

VyJ(v*)(t) < E j I d\{s)\T t V 

/•oo r ( roo 

/ V y J(y*)(t)-El / d\(s 



Tt 



a.s., t > 0, 



-St 



dv*(t) = 0, P-a.s., 



(4.3) 

(4.4) 

(4.5) 
(4.6) 



Ej / (0(t) -!/,(*)) dA(t)| =0, (4.7) 

for some nonnegative optional random measure o!A(w,£) such that E{L o!A(s)} < +oo. 

Lemma 4.1. For almost every uj € fi one /ias [iJ^ (X(w,i),0(w, t)) — 5] l{ v ,(^,)=e(w,-)}(0 ^ 0. 

Proof. Fix £ > 0. Then, for any stopping time n > t a.s., equation (|4.2p and the decreasing 
property of R y in its second argument imply that 



v*(t) <9(t), P-a.s., Vt > 0, 



hence 



e~ st < 



E 



e{ f 1 e~ Ss R y {X(s), sup Z(u))ds Ftl+E 

I Jt t<u<s J 



-<5ti 



F t \ a.s., 



e- ds Fj / (X(s),/(i))ds 



F, 1> >EU~ St -e~ ST1 



Tt \ a.s. 



In particular, for e > 0, define T\{e) := inf{s >t:R y (X(s),l(t)) > R y (X(t),l(t)) + e} to obtain 
-n(e) 



e| r ' e"^ (*(*)>*(*)) rfs|^t] < ^(F y (X(£),/(i)) + e)E{e- <5t -e- <5ri ^|F i } 



a.s.; 



that is, Ry(X(t),l(t)) + e > 5 a.s. for all e > 0. It follows Ry{X{t),l(t)) > 5 a.s., and 
hence [Ry (X(t),6(t)) — 5] !{;(■)>«(■)}(*) > a.s. for all t > 0, which provides the result since 
v*(oj,t) = 9(uj,t) is equivalent to l(u,t) > 0(uj,t) for almost every to € fi. D 

Theorem 4.2. // £/ie 6ase capacity process l(t) has continuous paths, then v*(t) (cf. |^. 1\ )) is 
optimal and the Lagrange multiplier d\(t) is absolutely continuous with respect to the Lebesgue 
measure. 

Proof. It sufficies to check the Generalized Kuhn- Tucker conditions (|4.4p - (|4.7|) for v*(t). Ob- 
viously v*(t) satisfies (|4.6p . Recall that the available resources process 6(t) is increasing and 
left-continuous. To show (|4.4|) and (|4.5|) . fix r G F, set tq := r, and recursively define 



r 2ri := inf{s > T 2n -i : l(s) < 0(s+)} 
T 2n+ i : = inf{s > T 2n : Z(s) > 9(s)} 



(U 
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with the convention inf{0} = +00. Notice that time T2 n +i, n > 0, is a time of increase for l(t). 
Then 

v*(s) = 0(s) for s G (t 2 „+i, r 2n +2], 



and 



l>*(s) = SUp Z(li) for S G (T2n,r2„+l] 



by the continuity of l(t). Moreover we have l(s) < 6(s) iov s £ (r, t\], hence sup T<u<s (l(u) A 9(u)) 
snp T < u<s l(u). 

Recalling (|4.ip and the previous considerations we have 



E 



e- 5s Ry(X(s), i/*(s)) ds 



Jr 



E 



e^i^X^),^))^ 



•7v 



+E E 



< E 



n=l 
n 



Tn + l 



e- Ss Ry(X(s),v*(s))ds 



e- ds R y (X{s), sup l(u)A6(u))ds 

T<U<S 



+ E E< ! / e- <5s J Rj / (X(s),^(s))(is 



e- 5s i? s/ (X(s), sup l{u))ds 

T2n<U<S 



n=l *• ^n-l 



•Ft 
Fr 



(4.9) 



+E E 



n=l 



-Ft 



where the equality holds if and only if r is a point of increase for v* . By definition of T\ , from 
dMD we get 



E 



e^ ds R y (X(s),^(s))ds 



Ft 



-5s 



<E<^ / e-° s R y (X{s), sup l(u))ds 



+E E 

n=l 

00 

+E E 



T2n 



r<M<s 



-8s 



T T 



(4.10) 



e -ds iij,(X(s),0(s))ds 



n=l 



T2n-1 

T"2n+1 

T2ii 



F r 



e^ ,5s ^(X(s), sup l(u))ds 



T2n<U<S 



Ft 



Since t\ and all odd indexed stopping times are times of increase for the process l(t), hence 
sup r < u<s l(u) = sup 'ti< u« Z ( n ) for s > r u and sup T2n < u<s l(u) = sup T2n+1 < u<s l{u) for s > r 2re +i- 
Therefore, from ()4.10p the stochastic backward equation (|4.2p implies 



E 



e~ ds ^(X(s),^(s))(is 



TV ^ = E<^ e -5T - e 



-«5n 



T T 
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+ ^E<| / " e~ 5s R y {X(s),9{s))ds 



Tr 



n=l *• ■ / l r 5»-l 
oo 



•Ft 



+E E 



n=l 



g — <5r2n _ g — <5t2„ + 1 



T2>i 



+ E E 1 / e- ds [^(X(s),0(s))-<5]ds 

71=1 ^ ^ T 2n-1 



Fr 



E 



e- ds [R y (X(s),9(s)) - 8\l {Vm=g} (s)ds 



Ft 



Notice that the process e [R y (X(t),9(t)) — S] 1/^=0} (t) is nonnegative by Lemma 14. II and it 
is Ft <S> B([0,t])— measurable. Hence, we set 



-St 



d\(t) := e- M [Ry(X(t),6(t)) - S\l {Vt=e} (t)dt 



(4.11) 



and we show that it is the optional measure Lagrange multiplier. Let us start by showing that 
d\(t) is an optional random measure on K + . That is, the continuous, increasing process 



(4.12) 



A(t) := / d\(s) 
l[0,t) 



is adapted to the filtration {J 7 t}t>o- Assumption 12. II and concavity of R in the second argument, 
imply that 

E{A(t)} = e| J e- 5s [R y (X(s),6(s))-5}t {l/t=e} (s)ds 

< e{ J e~ Ss R y (X(s),0 o )t {Ut=e} (s)ds\ 

< e{ f e- Ss R y (X(s),9 )ds\ 

< — e{ / sup R(X(s), u(s)) ds 1 < +oo. 

0o I Jo u(s)£S g J 



(4.13) 



Hence A(t) is dP-integrable and e~ 5t [R y (X(t),9(t))-5]t {Ut=e} (t) is dP®di-integrable on OxR + . 
Therefore, by Fubini's Theorem, the application u) — > A(tj,t) is ^-measurable and hence A is 
adapted. Then it is predictable since it is continuous. 

It follows that (|4.4p and (|4.5p hold and hence the process ()4.ip is optimal by Theorem l3.3l D 

Remark 4.3. The usual interpretation of the Lagrange multiplier as the shadow price of the 
value function may be heuristically shown as follows. After an integration by parts on the cost 
term, we may write the value function as 



V(9) = K< t I 



-Si 



R(X(t), sup (l(s) A 9{s))) - 5 sup (l(s) A 0(a)) 

0<s<t 0<s<t 



dt\ . 



Now, if v*(t) = sup 0<s<t (^(s) A 6(s)), then tr^-gy is the derivative (in some sense) of v* 
with respect to 9. We thus expect that the 'derivative' of V with respect to the constraint 9 is 
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e [Ry(X(t),9(t)) — 5]TL( Ut= 0\(t), which is exactly the density of the Lagrange multiplier in the 
case of a continuous 'base capacity' I (£). 

Proposition 4.4. The process Q(t) := E{A(oo) \Tt\ is a uniformly integrable martingale. 

Proof. By Assumption 12.11 the random variable A(oo) (cf. (|4.12|) ) is <iP-integrable. Hence the 
process Q(t) is a uniformly integrable martingale. □ 

Proposition 4.5. The process 

U(t):=R< [ dX(s) T t \ =E{A(oo)\J r t }-A(t) = g(t)-A(t) (4.14) 

is a supermartingale of class (D) and Q(t) — A(t) is its unique Doob-Meyer decomposition. 

Proof. Recall (cf. proof of Theorem I4.2p that the process A(t) of (|4.12p is increasing, adapted, 
continuous and integrable. Then lA{t) is cflP-integrable. Moreover, being d\ nonnegative, for 
s < t we have M{U(t) \J~ S } < U{s), i.e. U{t) is a supermartingale. Assumption 12.11 guarantees 
that it belongs to class (D). Hence Q{t) — A(i) is the unique Doob-Meyer decomposition of the 
supermartingale U{t) and therefore the process A(t) is the compensator oilA{t). □ 

If S(z^) is the Snell envelope of the supergradient V y J{v), i.e. 

S(i/)(t) = ess sup E{V y J(v)(T)\Tt}, (4.15) 

t<T< + 00 

then [5], Theorem 1, claims that the optimal investment plan z/* is characterized by the following 

conditions 

v* is flat off {V y J(v*) = §(z^)} (A-\f,\ 

A(y*) is flat off {u* = 9}, [ } 

where A{y*) is the predictable increasing process in the Doob-Meyer decomposition of the super- 
martingale §(i/*). Moreover §(z/*)(i) = E{yl(oo) - A{t) \T t } since V ?/ J r (i/*)(oo) = 0. If (I3TT21 
holds, then (|4TTSj) . g3D and dOJ imply that 

W(t) = S(i/,)(t) (4.17) 

at times of investment (when A{y*) and dX are not flat). This argument allows an enlighten- 
ing interpretation of the increasing, predictable, integrable process A(£). In fact at times of 
investment 

g(t) - A{t) = E j r d\{s) I F t \ = S(v*)(t) = M{u*){t) - A{v*)(t), (4.18) 

where A4(v*) is the martingale process in the unique Doob-Meyer decomposition of S(f*). By 
uniqueness 

e| I d\(s) Jij =M(v*)(t) and A(t) = A(u m )(t), (4.19) 

hence 

dA{v*)(t) = d\(t), P - a.s., Vt > 0. (4.20) 
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Therefore the second first order condition of (|4,16p coincides with the Kuhn- Tucker condition 
(|4.7p ; that is the Lagrange multiplier acts only when the constraint is binding. 

When l(t) is continuous, the explicit form of the Lagrange multiplier is known (cf. ([4. lip ), 
hence the compensator A(i/*)(t) is known as well. It follows that its paths are absolutely con- 
tinuous with respect to the Lebesgue measure and the Radon-Nykodym derivative of dA(v*)(t) 
ise-^(X(i),0(i))-5]l K=< , } (i). 

4.2 iV Firms: Finite Fuel and Operating Profit of Cobb-Douglas Type 

In the setting of Section [21 with T = +oo, we consider the Social Planner optimal investment 
problem (|2.4p for a market with N firms endowed with operating profit functions of Cobb- 
Douglas type, i.e. J?W (x,y) = ^i^""' with a { G (0,1), i = 1,2,..., N. 

Suppose that the economic shock process X(t) is given by X(t) = exp {Y(£)} for some Levy 
process Y(t) such that Y(0) = and with finite Laplace transform. Then (cf. [27], Proposition 
7.1) 

l {l) (t) = kiX(t), i = l,2,...,N, (4.21) 



with 



-oc 



ki=lE< e St eai ini < u<t Y(u) dt [ j ^ i = l,2,...,N, 

is the unique optional solution of the stochastic backward equation 

e{ [°° e- 5s R^(X(s), sup l®(u))ds T T \ = e~ Sr , Mt^T. (4.22) 

I Jt t<u<s J 

Define the optional process 






a(*) : = ^n :,L., - ( 4 - 23 ) 



Here /%(£) may be thought as the fraction of desirable investment of the i-th firm. By (j4.21|) . 



for t > and i = 1, 2, ..., N, we have that /%(£) is constant in time; in fact fait) = -=^ — =: /3j. 

2Jj=1 k 3 



.S (JUllSlclill 111 UI11B, 111 IclUl Pi\^ ) = — ' 

Fix t £ T and introduce the random times 

( ri(r)=inf{ S >r:E J =i^ ) ^)>^)} 



(4.24) 
<t 2 (t) =inf{s >r :^»(s) > ft0(s), Vi = 1,2, ...,iV}. 

Lemma 4.6. For all t GT we have o~i(t) = o~2(t) T-almost surely. 

Proof. Notice that IjPIll implies o"i(r) = inf{s > r : X(s) > J!v (g) fc . } = inf{s > r : jfejX(a) > 
#00), Vt = l,2,...,JV} = <r 2 (r). * =1 ' D 

Remark 4.7. If t £ T is a time of investment for all firms, that is dv* (t) > for all i, then 
the first Kuhn-Tucker condition in \3. 9\) guarantees that 

e{ j " e~ Ss R^(X{s),u < i\s))ds\T T \=W.{ j " e- 5s R^(X(s), i>i j \s)) ds | T T j. 
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Notice that if X is continuous, then ft' is continuous too due to (|4.2ip . 
Theorem 4.8. If the shock process X(t) is continuous then the process v_^ with components 

,( l )U\— o,,^ tl(i)(n,\ A R.Oln.W \l a.(i) i — T O AT (4-25) 



j/W(t)= sup (*«(«) A &0(u))Vy«, i = l,2,..,iV, 

0<w« 



is optimal for problem \2.J$ . Moreover, the Lagrange multiplier d\(t) associated to \2.1$ is 
absolutely continuous with respect to the Lebesgue measure. 

Proof. Let us check that v£ (t) satisfies the first order conditions of Theorem 13,31 Obviously 

E*Li v *Ht) < e ( f ) a - s - for a11 1 > °- 

The arguments of the proof are similar to those in the proof of Theorem 14.21 Fix r G T, set 
ro := t and define the sequence of stopping times r n as in (|4.8p but with J2i=i ' instead of I; 
that is, 

f r 2n+1 := inf{s > r 2n : ^£i * (<) («) > #(*)} 

\ r 2n+2 := inf{s > r 2n+1 : £^i ^ } (*) < 0(s+)}. 

Notice that the continuity of ft' implies 

v*(s)= sup Z W (u) for s G (r 2n ,r2„. 



(4.26) 



T"2n<«<S 



+ 1- 



Also r 2n +i = cri(r 2n ) = <r 2 (T 2n ) by Lemma H~6l hence r 2n .+i is a time of increase for all ft'. It 
follows 

4 (s) = PiO{s) for s G (r 2n+ i,r 2n+2 ]. 

Fix i = 1,2, ..., TV, and consider E{ f^° e~ Ry (X(s), v* (s))ds \T T }. Split the integral into two 
integrals f T1 and J°°. Since t\ is a time of increase for every ft', Remark ()4.7p holds and we 
may write 



E 



e- Ss R v l \X(s),vi l} (s))d 



T T \ = E 



"^(X^i/W^d 



■Ft 



+ E<j 



-^^(^w^* W) d 



-M+E 



-5s 



£&i#)(x(s),i/«(s))d 



(4.27) 



T T 



i¥=i 



since J> C J- Tl . Now, as in the proof of Theorem 14.21 we use the stopping times T n to split the 
last two integrals above and by the backward equation (|4.22|) corresponding to ft'(t) we may 
write 



E 



- Ss R^(X(s),ui i \s))ds\^ T 



(4.28) 



< e~ dT + E 



v 
[Y / ^ ) (X(s),/3 i 9(s))-8 



i=i 



1 {E ^„« =S} (# 



F T 



,«, 



with equality if and only if dul (r) > 0. Hence 

v 
-5t\ 

" 8=1 



p{t) := e- 5t [Y,Pi(R?(X(t),PAt)) ~ S) 



1 {E£i^°=«} ( * ) 
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is nonnegative by Lemma [4. II We may now define the Lagrange multiplier for the iV-firms Social 
Planner problem by dX(t) := p(t)dt since such dX is a nonnegative optional measure as in the 
proof of Theorem 14.21 □ 

Remark 4.9. For general operating profit functions satisfying Assumption \2.1\ we expect the 
solution for the Social Planner problem [2.$ to be 

u^(t)= sup {l®(u)APi(u)0(u))Vy®, i = l,2,...,N, 

0<u<t 

with 

/«(£) 

4.3 Constant Finite Fuel and Quadratic Cost 

Here we consider a monotone follower problem with constant finite fuel similar to those studied 
by Karatzas ([23]) |22j). and Karatzas and Shreve [21] (among others). In particular we discuss 
the example (cf. [5]) of optimal cost minimization for a firm that does not incur into investment's 
costs and has a running cost flow given by the convex function c(x, y) = ^(x—y) 2 of the economic 
shock x and the investment y. That is, we study the constrained convex minimization problem 

inf C(i/):= inf e{ f°° 8e- s 'hw(t) - v(t)f dt) (4.29) 

where W(t) is a standard Brownian motion and O is the positive constant finite fuel such that 
v(t) < 9 , P-a.s. for all t > 0. 

We expect to find a nonpositive Lagrange multiplier. Notice that 

V„C(i/)(t)=EJ I 5e~ Ss {v{s)-W{s))ds T t \. (4.30) 

Moreover, the backward equation 

e{ f 6e~ 5s sup l{u)ds jF T \ = e~ Sr W(r), Vr G T, (4.31) 

L Jt t<u<s ) 

is uniquely solved by the base capacity 

l(s) = W{s) - c, (4.32) 

where c is the positive constant c := E{ J °° 5e~ Ss sup 0<u<s W(u) ds}, by independence and 
time- homogeneity of Brownian increments. 

From [5] we know that the optimal investment policy is 

z/*(i)= sup ((W(s)-c)A6 o )Vv(0), (4.33) 

0<s<t 

which is the well known strategy of reflecting the Brownian motion at the threshold c until all 
the fuel is spent (cf. [22J). 
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We may write the subgradient (|4.30p at v* as 



V„C(^)(*) 



5e~ 



W*\s) 



W(s)) ds 



8e- 5s (v*(s)-W(s)) ds 



E 



-E 



Se 



Se~ os sup (W{u) -c) ds F t } + e~ ot W{t) 

t t<u<s 



-Sti 



-8s 



v*(s) — sup (W(u) — c) 

t<u<s 



ds 



Ft 



where we have used (|4.31|) in the second equality with I given by (|4.32p . With this trivial trick 
we are in the same setting as [5], proof of Theorem 2. Hence we have that the Snell envelope of 
the subgradient evaluated at the optimum u* (cf. (|4.33p ) is 



S(i/*)(t) =E 



6e 



v*(s) — sup (W(u) - c) 

t<u<s 



AOds 



Tt 



(4.34) 



or, equivalently 



S(i/*)(t) =E 



Se 



-(5s 



re (t) 



6 - sup (W(u) - c) 

t<u<s 



ds 



F, 



with 



T do (t):=mi{s>t:W(s)-c>e o }, 



(4.35) 



by means of (|4.33p . Notice that Tg (t) is a time of increase for W(t) — c. Hence we have 
su Pt<u<s( w ( u ) ~ c ) = sn Prg (t)<u<s( w ( u ) - c ) for s e (Te (t), +oo\. Therefore (|4.31|) implies 



S(i/*)(t) =E 



<5e~ 



? cfo 



**„(*) 



J- f }-E{e-^«I^ (i))|^}; 



that is, 



S(i/,)(t) = E{ e-^oi*) ^ _ W (r 9o {t))} |^}- 



(4.36) 



Now we first find the explicit form of the Snell envelope S(^*)(i), and then we use it to 
identify the compensator part of its Doob-Meyer decomposition; that is, the Lagrange multiplier 
of problem (|4.29p (cf. (|4.20p ). We start by showing that S(v*) is an ^-martingale until the 
base capacity l(t) = W(t) — c is below the finite fuel 8 ; that is, {S(z/*)(it A Tg (t)} u >t is an 
J^-martingale. In fact, by [26], Corollary 3.6 it suffices to prove that {§(z/*)(u /\Tg (t)} u >t is an 



F 



uAt 6o (t) 



•martingale, and that follows by iterated conditioning and the fact that tq o {u\ /\Tg (t)) 



t~9 {u2 A Tg o (t)) for all Ui, «2 £ [t,oo). Next, we define 

<T $0 (t) := inf{s > t : W(s) < 9 + c}, 

and we show that S(i/*) is an J-^-submartingale when the base capacity l(t) is above the finite 
fuel O ; that is {§>(i^)(u AaQ o (t)} u >t is an -A«-submartingale. Again, as above, it suffices to prove 
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that it is an an T uAag (^-submartingale. In fact, from Tg o (u A cro (t)) = u A (Tg (t) for all u>t, 
the martingale property of W and 5e~ 5s W(s)ds = —d(e~ Ss W(s)) + e~ 5s dW(s), follows that 
the process S(V*)(u A o~g o (t) + J t " ae ° 5e~ Ss (9 — W(s))ds is an T u/ \ ae (^-martingale, hence an 
./-"^-martingale. Therefore S(i/*)(u A 0~$ o (t)) is an J^-submartingale with absolutely continuous 
compensator A(y*) given by 

dA(v*)(s) :=-5e~ 5s {9 - W(s))ds, s € [t,u Aag o {t)],u>t, (4.37) 

since W{-) < O on [t,u A crg (t)], u>t. 

Now, as the Lagrange multiplier (|4,lip acts only when v*(t) = o , i.e. only when l(t) > o , 
we conclude that the Lagrange multiplier of problem (|4.29|) is 

d\(t) = 5e~ 8t [d - W(t)} t m . )>9o+c} (t) dt, (4.38) 

which, as expected, is negative and coincides with the opposite of the optional measure dA(i/*)(t) 
(cf. (JQ7JI ). 

Remark 4.10. In ^ Benes, Shepp and Witsenhausen considered a problem with the same cost 
functional but they allowed controls of bounded variation. 

4.4 Constant Finite Fuel and Operating Profit of Cobb-Douglas Type 

We consider the maximization problem of profit, net of investment costs, 

sup J(i/) := sup E / e- Ss R(X(s),v(s))ds- e~ Ss dv(s)\. (4.39) 

v£Se ^es 9o I Jo Jo J 

The finite fuel is given by the positive constant O , hence the controls satisfy < v(t) < O 
P-a.s., for all t > 0. The economic shock process X(t) is modeled by a Geometric Brownian 
motion 

X(t) = x e^-^ a2)t+aW ^ with x > 0. (4.40) 

The firm's operating profit function is of the Cobb-Douglas type and depends on the economic 
shock x and the investment policy y; i.e., R(x,y) = j^ -x a y 1 ~ a with < a < 1. As pointed 
out in [27] this construction is consistent with a competitive firm which produces at decreasing 
returns to scale or with a monopolist firm facing a constant elasticity demand function and 
constant returns to scale production. Notice that problem (|4.39p has been studied in detail in 
[25j in the case of O = +oo by a dynamic programming approach. 

It is known (cf. [5]) that the unique optimal solution for problem (|4.39p is given by 

i/»(t)= sup (Z(s)A0 o )Vi/(O), (4.41) 

0<s<4 

where the optional process l(t) uniquely solves the stochastic backward equation (cf. [1]) 

e{ r e- 5s X a (s)( sup l{u))~ a ds T T \ = e~ 5r , Vt£T. (4.42) 

I Jt t<u<s ) 
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As shown in [27j, Proposition 7.1, when the shock process is of exponential Levy type, i.e. 
X(t) = XQe Y ^\ with Y(t) a Levy process such that 1^(0) = 0, then the solution of ()4.42p is given 
by the base capacity 

l(t) = kX(t), (4.43) 

where k = (^E{e Q — ( r ( "})~, Y_(t) '■= info< u <t Y(u) and t(5) is an independent exponentially 
distributed time with parameter 5. 
From (|4.39p we have 



V v J(i/)(t)=EJ f" e~ 5s X a 



(s)v~ a (s)ds 



Ti 



-st 



(4.44) 



Following [5], proof of Theorem 2, we know that the Snell envelope of supergradient (|4,44p 
evaluated at the optimal control policy (|4.4ip is 



= E 



-5s 



X a (s) ( ( sup (kX(u) A d ) V i/(0) 

0<u<s 



sup kX(u 

t<u<s 



ds 



(4.45) 

r,\. 



Fix t > and define the stopping time 

r (9o (t):=inf{s>t:fc-X'(a)>0 o }. 



(4.46) 



It is a time of increase for X(t). Now we split the integral into f^ e ° +1°°^, then the first 
one vanishes due to (|4.46j) and we are left with 



S(i/*)(t) = E< 



Q E 



X a (s)((0 o )- a - ( sup kX(u)r a )] ds Iji 
( r e~ 5s X a {s)ds Jil-Efe-^^Iji} 



where we have used (|4.42|) to obtain the second equality. 
Lemma 4.11. Assume S > \x + cr 2 . Then for every t > 0, one has 



E 



l JrgJt) J 



1— -E{e- fr «»«r(r,(t)) T t \. (4.47) 



i T9o (t) ) (5 - fia) + 2<7 2 a(l - a) 

Proof. The proof follows from the Markov property and the Laplace transform of a Gaussian 
process. Independence of Brownian increments, together with W(u+Tg o (t)) — W(Tg (t)) ~ W(u), 
allow us to write 

POO 



E 



( r e - 5s x a (s)ds Ft] =e(e( r 

= e| e' <5r9 ( ' ) X a (r eo (t))EJ /°° e^V^a 
= e| e- <5Te ° (t) X a (r eo (i))E| f°° e' Su e a ^-^ 



e- ds X a {s)ds 






re {t) 



Ft 



a*)u+ao(W(u+T Bo (t))-W(rg (*))) ^ 



•Ft 



a 2 )u+aaW(u) . 



7i 



e<| e -^(')r(T«jt)) / e -(<5-/^)«-i<xMi-«)«d n 



t; 



Notice that (<5 — [ia) + ■^a 1 a(\ — a) > by the assumption, hence (J4.47J) follows. 



□ 
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Now Lemma 14.111 and (|4.47p imply 

8 M® = (5 - ^)+° i~2 a (i - a) E {^° (t>r ^(^ |^i} -E{e-*»-W |^}. (4.48) 

By arguments similar to those used in Subsection 14.31 we find the explicit form of the Snell 
envelope S(i/*)(t) and hence we identify the compensator part of its Doob-Meyer decomposition 
as the Lagrange multiplier of problem (|4.39p . In fact, we have that S(v*) is an J-^-martingale 
until the base capacity l(t) = kX(t) is below the finite fuel O , since rg o {u\ A Tg (t)) = tq o (u2 A 
rg (t)) for all ui, U2 £ [t, oo). Then, if 

ae (t) := mi{s > t : kX(s) < 6 }, 

we show that §(i/*) is an J^-supermartingale when the base capacity l(t) is above the finite 
fuel O ; that is {S(^*)(u A crg o (t)} u > t is an J^-supermartingale. It suffices to prove that it is an 
J r uAag ( t )-supermartingale (cf. (2B], Corollary 3.6). In fact, from Tg o (u A crg (t)) = u A (Tg (t) for 
all u > t and d(e~ Ss X a (s)) = -5e~ Ss X a (s)ds + e- &s dX a {s) follows that the process §(z/*)(u A 
a o (t) + f™ ae ° e~ Ss (X a (s)(9 )~ a — S)ds is an T uN(Je (t) -martingale, hence an J^-martingale. 
Notice that X a {s){6 )~ a > S for all s € [t,u A <r(t)). In fact, we have X a (s)(# )" a > fc~ Q 
for s e[t,uA a{t)) since fc = (±E{e a ^ T ( 5) )})« , y(«) := inf < s < u [(/i - 5CJ 2 ) s + aW(s)] , and 
E | e ay(r(5))} = p_(p_ - a)~ l < 1 with /3_ the negative root of ±cr 2 x 2 + (// - \a 2 ) x - 6 = 
and a > 0. Therefore, §(z/*)(u A crg a (t)) is an J^-supermartingale with absolutely continuous 
compensator A(z/*) given by 

dA{y*)(u) := e~ 5u (X a (u)(6 o y a - 8) du. (4.49) 

Finally, as the Lagrange multiplier optional measure dX (cf. (|4.11|) ) acts only at times when 
v*(t) = do (he., only when l(t) > O ), we conclude that for problem (j4.39p . dX must be 

dX(t) = e~ st {X a (t)(0 o r a - 5) l {kX (.)>e o} (t) dt; (4.50) 

that is, dX(t) coincides with the random measure dA(v*)(t) (cf. (|4.49p ). 
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